Systems and methods for categorizing, evaluating, and displaying user input with publishing content

ABSTRACT

Systems and methods are provided for displaying received publishing content on a web page along with one or more user elements by which one or more users may submit sentiment input or textual input in relation to the received publishing content or a subportion of the received publishing content; receiving a user input related to the displayed publishing content displayed on the web page, the user input including an identification of a subportion of the displayed publishing content and a sentiment input or a textual input; analyzing any user input received from each of one or more of the plurality of users in relation to the subportion; computing a sentiment score based on analysis of the analyzed user inputs received from each of one or more of the plurality of users in relation to the subportion; and displaying indicia representing the sentiment score computed for the subportion.

TECHNICAL FIELD

Various embodiments of the present disclosure relate generally toprocessing electronic messages, such as over the Internet. Morespecifically, particular embodiments of the present disclosure relate tosystems and methods for processing, evaluating, and displayinguser-generated sentiment input and user-generated comments related toweb pages and subportions of web pages.

BACKGROUND

Typically, online publishers, such as online media companies and otherpublishers of articles, stories, and other electronic content, provideonline web page space and mechanisms for viewers to comment on, orotherwise interact with, that published content. Some published contentattracts many comments, which may be too numerous for a reader to easilyreview and digest. Large sites with lots of traffic can receivethousands of user comments in relation to a single popular article. Theprevalent way to present comments related to the articles is to presentthem together at the end of the article, which may not be ideal for allreaders. Often, the comments themselves are as interesting as thecontent, but reading through all of them to find particular takes oropinions is not practical.

In some cases, comments may relate only to one subportion, e.g., aparticular paragraph, sentence, fact, etc., of the article. Also,specific claims or facts cited in articles may be of special interestand may spark robust debate, but comments specific to such points cannotbe discerned from general comments about the article. Because commentsare typically displayed in sequential order, it can be difficult forreaders to identify comments related to specific topics or specificsubportions of interest to the reader. Due to the standard location andscope of comments, comments of interest to a particular user may be toofar down a chain of comments to appear within a useful distance from theoriginal content.

Similarly, some online media companies allow users to rate or provideother forms of expressing sentiment regarding an article as a whole, butno ability to express sentiment or view the overall sentiment related tospecific subportions or specific user comments.

SUMMARY OF THE DISCLOSURE

According to certain embodiments, computer-implemented methods aredisclosed for evaluating user input relating to electronic publishedcontent. In an exemplary method, the method includes: receiving, over anelectronic network, electronic publishing content for display online;displaying the received publishing content on a web page along with oneor more user elements by which one or more users may submit sentimentinput or textual input in relation to the received publishing content ora subportion of the received publishing content; receiving, from each ofa plurality of users, a user input related to the displayed publishingcontent displayed on the web page, the user input including anidentification of a subportion of the displayed publishing content and asentiment input or a textual input; analyzing, for each subportion ofthe displayed publishing content, any user input received from each ofone or more of the plurality of users in relation to the subportion;computing, for each subportion of the displayed publishing content, asentiment score based on analysis of the analyzed user inputs receivedfrom each of one or more of the plurality of users in relation to thesubportion; and displaying, for each subportion of the displayedpublishing content, indicia representing the sentiment score computedfor the subportion, along with at least one user element by which one ormore further users are enabled to provide user input to further modifythe computed and indicated sentiment score.

According to certain embodiments, systems are disclosed processing,evaluating, and displaying user input. One system includes a memoryhaving processor-readable instructions stored therein; and a processorconfigured to access the memory and execute the processor-readableinstructions, which when executed by the processor configures theprocessor to perform a method. In an exemplary method, the methodincludes: receiving, over an electronic network, electronic publishingcontent for display online; displaying the received publishing contenton a web page along with one or more user elements by which one or moreusers may submit sentiment input or textual input in relation to thereceived publishing content or a subportion of the received publishingcontent; receiving, from each of a plurality of users, a user inputrelated to the displayed publishing content displayed on the web page,the user input including an identification of a subportion of thedisplayed publishing content and a sentiment input or a textual input;analyzing, for each subportion of the displayed publishing content, anyuser input received from each of one or more of the plurality of usersin relation to the subportion; computing, for each subportion of thedisplayed publishing content, a sentiment score based on analysis of theanalyzed user inputs received from each of one or more of the pluralityof users in relation to the subportion; and displaying, for eachsubportion of the displayed publishing content, indicia representing thesentiment score computed for the subportion, along with at least oneuser element by which one or more further users are enabled to provideuser input to further modify the computed and indicated sentiment score.

According to certain embodiments, a non-transitory computer readablemedium is disclosed as storing instructions that, when executed by acomputer, cause the computer to perform a method, the method receiving,over an electronic network, electronic publishing content for displayonline; displaying the received publishing content on a web page alongwith one or more user elements by which one or more users may submitsentiment input or textual input in relation to the received publishingcontent or a subportion of the received publishing content; receiving,from each of a plurality of users, a user input related to the displayedpublishing content displayed on the web page, the user input includingan identification of a subportion of the displayed publishing contentand a sentiment input or a textual input; analyzing, for each subportionof the displayed publishing content, any user input received from eachof one or more of the plurality of users in relation to the subportion;computing, for each subportion of the displayed publishing content, asentiment score based on analysis of the analyzed user inputs receivedfrom each of one or more of the plurality of users in relation to thesubportion; and displaying, for each subportion of the displayedpublishing content, indicia representing the sentiment score computedfor the subportion, along with at least one user element by which one ormore further users are enabled to provide user input to further modifythe computed and indicated sentiment score.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments. The objects and advantages of the disclosedembodiments will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the scope of disclosed embodiments, as setforth by the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate various exemplary embodiments andtogether with the description, serve to explain the principles of thedisclosed embodiments.

FIG. 1 is a schematic diagram of a network environment for processingand displaying user input with published content, according to anembodiment of the present disclosure.

FIG. 2 is a flow diagram of an exemplary method for analyzing user inputrelated to subportions of published content and displaying indicia basedon the analysis, according to an embodiment of the present disclosure.

FIG. 3 illustrates an exemplary graphical user interface (GUI) of a webpage displaying published electronic content, according to an embodimentof the present disclosure.

FIG. 4 illustrates an exemplary GUI of an excerpt of a web page alongwith user elements for receiving user input, according to an embodimentof the present disclosure.

FIG. 5 illustrates another exemplary GUI of an excerpt of a web pagealong with user elements for receiving user input, according to anembodiment of the present disclosure.

FIG. 6 illustrates another exemplary GUI of an excerpt of a web pagealong with user elements for receiving user input, according to anembodiment of the present disclosure.

FIG. 7 illustrates another exemplary GUI of an excerpt of a web pagealong with user elements for receiving user input, according to anembodiment of the present disclosure.

FIG. 8 illustrates another exemplary GUI of an excerpt of a web pagealong with user elements for receiving user input, according to anembodiment of the present disclosure.

FIG. 9 is a block diagram of an exemplary computer system in whichembodiments of the present disclosure may be implemented.

DESCRIPTION OF THE EMBODIMENTS

While the present disclosure is described herein with reference toillustrative embodiments for particular applications, it should beunderstood that embodiments of the present disclosure are not limitedthereto. Other embodiments are possible, and modifications can be madeto the described embodiments within the spirit and scope of theteachings herein, as they may be applied to the above-noted field of thepresent disclosure or to any additional fields in which such embodimentswould be of significant utility.

In view of the challenges associated with the conventional techniquesoutlined above, systems and methods are disclosed herein for analyzingand displaying user sentiment and user textual input adjacent topublished electronic content. As described below, these challenges maybe addressed in a number of ways. First, as described in further detailwith respect to FIGS. 3 and 8, user inputs, e.g., comments, may beanalyzed to determine (1) a sentiment score for the user input based oncontextual analysis and/or other users' sentiments about said comment,(2) which subportion of a web page said comment is related to and/or (3)appropriate meta tags (“tags”) to assign to said comment. Second, asdescribed in further detail with respect to FIGS. 4-8, received userinput may be associated with and displayed in spatial relation to (e.g.,adjacent to) particular subportions of the published electronic content.Using various techniques, as further described below, each subportionmay be assigned appropriate tags and/or a sentiment score for generatingrelevant indicia to associate with the subportion.

Further, in some embodiments, the challenges described above may beaddressed by providing article-specific/customizable meta tags andfilters that enable the user to provide context to their comment (e.g.“pro-campaign finance reform,” “bad supreme court decision,” “citedreferences,” etc.). In some embodiments, the challenges described abovemay be addressed by enabling users to cite/remark on specificsubportions of the content (e.g., targeted comments “This quoted studywas debunked in the NE Journal of Medicine, see”). Additionally oralternatively, in some embodiments, the challenges described above maybe addressed by providing users the means to filter and locatecomments/commenters of a particular disposition. The comment meta-data,coupled with data the user opts to make public (i.e. via their profiletags, e.g. “Conservative,” “Liberal,” etc.) provides a more social andinteractive experience.

User input, as used herein, may include comments or messages, “likes,”“dislikes,” etc., submitted by different users, e.g., as part of anonline article, message board, or forum provided via a web pageaccessible over the Internet. The different users may be participants ofone or more virtual conversations or message threads including a seriesof comments posted by different users/participants at various times tothe online article, message board, or forum. The comments may beassociated with an article or a blog entry displayed on the web page.Each user may post or submit user input related to a specific subportionof the article or blog entry in this example. Subportions may be asection, title, paragraph, sentence, phrase, fact, word, etc. The userinput may be submitted by each user via an interface provided on a webpage that is loaded within a web browser executable at the user'scomputing device. Also, the user input may be anything, including, butnot limited to comments and/or sentiment. Comments may be in the form ofelectronic messages including, for example, text, graphics (e.g., iconsor “emoticons”), and/or other types of content (e.g., embedded audio orvideo content). Sentiment may be in the form of ratings, grades, thumbsup or thumbs down, and/or up votes or down votes.

In some embodiments, comments and sentiment may be displayed in closespatial relation to subportions of electronic publishing content.Indicia may be associated with each subportion indicating usersentiment. As will be described in further detail below, a sentimentscore may be calculated for a subportion based on one or more parametersassociated with user input. Examples of such parameters include, but arenot limited to, analysis of textual input, analysis of sentiment input,analysis of the submitting user's profile, a demographic of thesubmitting user, activity level of the conversation, and/or a number ofusers providing content related to the subportion.

In the detailed description herein, references to “one embodiment,” “anembodiment,” “an example embodiment,” etc., indicate that the embodimentdescribed may include a particular feature, structure, orcharacteristic, but every embodiment may not necessarily include theparticular feature, structure, or characteristic. Moreover, such phrasesare not necessarily referring to the same embodiment. Further, when aparticular feature, structure, or characteristic is described inconnection with an embodiment, it is submitted that it is within theknowledge of one skilled in the art to effect such feature, structure,or characteristic in connection with other embodiments whether or notexplicitly described.

Reference will now be made in detail to the exemplary embodiments of thedisclosure, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts.

FIG. 1 is a schematic diagram of an exemplary network environment inwhich electronic messages and other user input may be processed anddisplayed, according to an embodiment of the present disclosure. Asshown in FIG. 1, the environment may include a plurality of user orcommenter devices 102 that are communicatively coupled to each other aswell as a plurality of server systems 106, a browser web server 114,and/or a mobile web server 116 via an electronic network 100. Electronicnetwork 100 may include one or a combination of wired and/or wirelesselectronic networks. Network 100 may also include a local area network,a medium area network, or a wide area network, such as the Internet.

In one embodiment, each of user or commenter devices 102 may be any typeof computing device configured to send and receive different types ofcontent and data to and from various computing devices via network 100.Examples of such a computing device include, but are not limited to, adesktop computer or workstation, a laptop computer, a mobile handset, apersonal digital assistant (PDA), a cellular telephone, a networkappliance, a camera, a smart phone, an enhanced general packet radioservice (EGPRS) mobile phone, a media player, a navigation device, agame console, a set-top box, or any combination of these or other typesof computing devices having at least one processor, a local memory, adisplay (e.g., a monitor or touchscreen display), one or more user inputdevices, and a network communication interface. The user input device(s)may include any type or combination of input/output devices, such as akeyboard, touchpad, mouse, touchscreen, camera, and/or microphone.

In one embodiment, each of the user or commenter devices 102 may beconfigured to execute a web browser or mobile browser installed fordisplaying various types of content and data received from any of serversystems 106 and/or web servers 114 and 116 via network 100. Serversystems 106 in turn may be configured to receive user input, e.g., inthe form of comments, from user or commenter devices 102 over electronicnetwork 100. The comments may be submitted by a user at each device 102through an interface provided on a web page loaded within the browserexecutable at each device.

While browser web server 114 and mobile web server 116 are shownseparately in FIG. 1, it should be noted that web servers 114 and 116may be implemented using a single server device or system. In anexample, such a single server may be a web server that is configured toprovide different versions of a web page and associated content to eachof user/commenter devices 102 according to the type of device or webbrowser executable at the device. The different versions of the web pagemay include, for example, a desktop version and a mobile version, forwhich the web page content may be formatted appropriately for displayvia the particular type of browser at the device. Further, any of thedevices or functionality of server systems 106, browser web server 114,and/or a mobile web server 116 may be combined together or separated,and may be operated by a single administrative entity, or outsourced toone or more other entities, such as a web hosting entity, web storageentity, and/or cloud computing service, possibly disposed remotely ofeach other.

As shown in the example of FIG. 1, server systems 106 may include acommenting processor 110. In an embodiment, commenting processor 110 maybe configured to analyze and execute a scoring algorithm in order tocalculate or compute a sentiment score, as will be described in furtherdetail below. The sentiment score may reflect, for example, thepopularity, importance, feeling, and/or positivity/negativity of acomment and/or subportion based on the user input provided with respectto the comment and/or subportion, as received from user/commenterdevices 102.

Also, as shown in FIG. 1, server systems 106 may include one or moredatabases 108. In an embodiment, databases 108 may be any type of datastore or recording medium that may be used to store any type of dataincluding, for example, user input (e.g., comment, sentiment, etc.) aswell as the scoring algorithm used to score such content. In anembodiment, commenting processor 110 may be configured to receive userinput from user/commenter devices 102 and store the received contentwithin databases 108. In some implementations, any received data may bestored in the databases 108 in an encrypted form to increase security ofthe data against unauthorized access.

In a further embodiment, commenting processor 110 may be configured toprocess the user input using the scoring algorithm. In some embodiments,sentiment scores are calculated for a user input, for a relevantsubportion, or for both. For example, in some embodiments, the scoringalgorithm determines the sentiment by calculating a sentiment score forthe user input. Additionally or alternatively, sentiment expressedwithin user input may be used to calculate a sentiment score for thesubportion to which the user input relates. Further, in someembodiments, the sentiment scores calculated for user input may factorinto the scoring algorithm for the subportion for which they relate. Forexample, a subportion may have several comments related to it. Asentiment score may be determined based on an analysis of these commentsso a user can see how other users feel about said subportion. It may bebeneficial for the user to, additionally or alternatively, see otheruser opinions on these comments. For example, one comment may be a spamadvertisement and thus receive low ratings from other users, therebyresulting in a low sentiment score. This comment may be highlighted inred and, as described with respect to FIG. 8, since comments with lowscores may be filtered, it might not be displayed to the user at all. Insome embodiments, the sentiment score of the comment may then be used inthe scoring algorithm for the subportion. For example, the sentimentexpressed within a comment with a low sentiment score (e.g., a commentthat received low ratings from other users) may be given less weight inthe scoring algorithm for calculating the sentiment score of the relatedsubportion.

In a further example, commenting processor 110 may use the scoringalgorithm to compute a sentiment score for each user comment and/orsubportion within a web page or online message board or forum includedwithin the web page. As described above, each subportion may haveelicited one or more sentiments and/or comments posted or submitted bydifferent users of the message board or forum within the web page, asdisplayed at each user's device (e.g., each of user/commenter devices102). The computed sentiment score for each user input may be assignedto the subportion and used to calculate an overall sentiment score forthat subportion. Indicia based on the received input and/or computedoverall sentiment score may be displayed relative to the subportion. Asdescribed above, the sentiment score computed for each subportion mayreflect a level of popularity, agreement, importance, feeling, and/orpositivity/negativity, which may be determined based on variousparameters associated with the sentiment, comment, individualparticipants, and/or users/commenters providing user-generated contentrelated to the subportion. Accordingly, the subportions may be scoredsuch that subportions determined to be relatively more agreeable,popular, important, and/or positive/negative are given relatively highsentiment scores, and may be, for example, associated with indiciaidentifying relative agreeableness, popularity, importance, and/orpositivity/negativity.

Server systems 106 may also include a commenting user interface (UI)module 112 that facilitates receiving user input from users anddisplaying the received user input. The displayed content may include,for example, user input that has been processed or scored along with thesubportion to which it relates, as described above. For example,commenting UI module 112 may be configured to generate, render, andtransmit web page content including an article or blog entry. Such webpage content may be divided into subportions. Displayed within thewebpage may be one or more message streams of comments posted by usersrelated to some or all subportions. Thus, the web page content mayinclude user input in the form of comments/sentiment. The user input mayinclude displaying, for example, any one or combination of text, images,sentiment (e.g., icons for users to “like” or “pin” their favoritesubportions or comments, etc.) to user/commenter devices 102 via network100. Additional features and characteristics of the commenting andscoring functionality of commenting processor 110 will be described infurther detail below with respect to the exemplary graphical userinterfaces (GUIs) of comments and sentiments associated with subportionsof a web illustrated in FIGS. 3-8.

FIG. 2 is a flow diagram of method 200 for changing and displaying asentiment score indicating the user sentiment for a subportion ofreceived electronic publishing content of a web page, according to anembodiment of the present disclosure. It should be noted that, althoughthis exemplary method describes sentiment scores for subportions of aweb page, this method may be applied to user comments to identify asentiment score, display indicia, and/or tag a user comment or commentconversation. The method may involve algorithmically computing asentiment score for each subportion. For example, the score may be afunction of a number of factors, which may be represented as acombination of variables for a scoring formula. The score may be updatedperiodically or as desired by adjusting a value of each variable withinthe formula. In an embodiment, the score may be updated dynamically inresponse to detecting or receiving an indication of a new user actionwith respect to a subportion or a comment on the subportion. Such a newuser action may include, for example, the addition of user input (e.g.,a new comment, a new “thumbs up”) related to a subportion of theelectronic content. The user input may be received from a user or userdevice (e.g., any of user/commenter devices 102 of FIG. 1, as describedabove) via a communication network (e.g., network 100 of FIG. 1, asdescribed above).

As shown in FIG. 2, method 200 may include steps 202, 204, 206, 208,210, 212, 214, 216, and 218. However, it should be noted that method 200may include more or fewer steps as desired for a particularimplementation. In an example, one or more of the above-listed steps ofmethod 200 may be executed by server systems 106, browser web server114, and/or mobile web server 116 of FIG. 1, as described above.However, method 200 is not intended to be limited thereto, and the stepsof method 200 may be performed by any server or other type of computingdevice having at least one processor, a memory, and a networkcommunication interface for sending and receiving information from oneor more user devices.

As shown in FIG. 2, method 200 may include receiving electronicpublishing content for display online (step 202). Method 200 may alsoinclude dividing the received content into a plurality of subportions(step 204). In some examples, the electronic content for display onlinemay be an article or blog entry. In some embodiments, the receivedcontent may be divided into letters, words, phrases, sentences,paragraphs, and/or sections. In some embodiments, step 204 may not beperformed initially, and instead, subportions may be created once auser/users select a portion of the electronic content for comment orsentiment. For example, a user may highlight a phrase he/she wishes tocomment on and then this highlighted portion may be designated as asubportion.

In step 206, a server, e.g., browser web server 114 and/or mobile webserver 116 of FIG. 1, may generate and display the received content on aweb page. In some embodiments, the indicia distinguishing subportionsmay be displayed during step 206 as well. Once the received content isdisplayed on the web page, a user may be able to provide input (e.g.,comments or sentiments) related to subportions of the web page. In step208, a server may receive such user input on at least one of thesubportions. The user input may be any type of input. The user input mayalso include an identification and/or designation of a user-specificsubportion of the electronic publishing electronic publishing content.For example, a user may select some words or sentences, may select anddrag, may highlight, etc. User input may be received from auser/commenter device 102 over the Internet.

FIG. 2 reflects that the received user input may include sentiment inputand/or textual input, but this disclosure is not limited thereto. Insome embodiments, if a sentiment input was received in step 208, method200 may proceed to step 210. Sentiment input may include receiving fromone user/commenter device 102 a “rating,” a “like,” a “favorite,” a“block,” an “up vote,” a “thumbs down,” or any other action forexpressing sentiment. The received sentiment input may be analyzed (step210). In some embodiments, the analysis may include determining whetherthe sentiment is positive or negative. For example, in some embodiments,a thumbs up or thumbs down icon may be available for sentiment input. Ifa user selects the thumbs up icon, an analysis may determine that thisis a positive input. In some examples, indicia of this sentiment may bedisplayed. There may be a count of each thumbs up or thumbs down. If thecurrent display shows 58 thumbs up and 43 thumbs down and user selectsthumbs down, the display may update to 58 thumbs up and 44 thumbs down.In one embodiment, sentiment may be a sliding scale of 1-5, the Likedscale, etc. Once the received sentiment input has been analyzed, method200 may proceed to step 216.

If, in step 208, the received user input is textual input (e.g. acomment) related to the at least one subportion of the web page, method200 may proceed to 212. In step 212, the received textual input may bedisplayed in relation to the subportion. For example, as shown in FIG.4, the user's textual input related to subportion 304 may be displayedin comment box 422. Method 200 may then proceed to step 214 or, in someembodiments, method 200 may skip step 212 and proceed to step 214 afterreceiving user textual input in step 208.

In step 214, method 200 may include analyzing the text for sentiment. Insome embodiments, a keyword search may be performed to determinesentiment. For example, if a negative word (e.g., “disagree”) isdetected within the textual input, the analysis may determine thesentiment of the textual input is negative. If strong sentiment terms orwords (e.g., “love”) are detected, the analysis may determine thesentiment of the textual input is highly positive or negative.Similarly, if there are multiple positive terms within the textualinput, the analysis may determine the sentiment of the textual input ishighly positive. Other analysis of the textual input may also includelength, punctuation, etc.

In some embodiments, the user's action history and/or user's profilemay, additionally or alternatively, be analyzed in step 214. Thesentiment of a user's textual input and/or appropriate tags for saidinput may be based on a user's profile or history by default. Forexample, if the user identifies as a liberal in his/her profile, textualinput related to a subportion praising conservatives may, by default, beanalyzed as being a negative sentiment. Similarly, if a user's historyshows repeated negative textual input when an article, comment, and/orsubportion praise liberals, all comments on subportions praisingliberals may, by default be analyzed as containing a negative sentiment.In some embodiments, a user may wish to hide words like “liberal” and“conservative” from being pulled from their profiles to analyzecomments.

Other examples of relevant statistics that may be used during theanalysis of the textual input may include, but are not limited to, thenumber of comments received (e.g., during a predetermined period oftime), the time-of-day when the comments are received, and parameters(e.g., geographic location, noteworthiness or popularity, commentingfrequency, etc.) of the users from which comments are received. In anembodiment, each user may be associated with a popularity index orrating. The popularity rating of a user may be based on, for example,the number of other users who may be listed as “fans” or “followers” ofthe user. Additionally or alternatively, the popularity rating may bebased on the number of “likes” or number of times that other users havemarked a comment posted by the user as a favorite or otherwise indicatedtheir approval of the user's comments. Other users may indicatethemselves to be fans of the user or mark a comment by the user as afavorite by selecting a user interface control element provided via, forexample, a GUI for displaying conversations and user-submitted commentsassociated with an online message board or forum.

In some embodiments, textual input may also be analyzed for appropriatetags. Analyzing comments for appropriate tags will be described infurther detail below.

After completion of step 214, method 200 may proceed to step 216. Instep 216, a sentiment score associated with the at least one subportionmay be modified. This modification may be based on the analysis in step210 of the received sentiment input and/or the analysis in step 214 ofthe received textual input. In some embodiments, a sentiment score maybe initially set at a neutral position (for example, 0 in a scale from−5 to +5, or 2.5 out of 5 stars) and each time a user providesfeedback/sentiment regarding that subportion, the sentiment score may beadjusted. After modifying the sentiment score associated with the atleast one subportion, method 200 may proceed to step 218. In step 218, aserver may change user sentiment indicia of the at least one subportionbased on the modified sentiment score. In some embodiments, if thesentiment score was decreased due to negative sentiment input or textualinput, a box encompassing the at least one subportion may turn from darkgreen to light green or a set of stars may go from 3.8 to 3.75 stars.

FIG. 3 illustrates an exemplary GUI for displaying received electroniccontent on a web page. Specifically, FIG. 3 depicts an entire article,“Changing Political Landscapes,” published on a web page. In theembodiment illustrated in FIG. 3, title 2 is displayed above a “ShowSentiment” icon 4 and “Hide Sentiment” icon 6. The body of the articleis displayed below “Show Sentiment” icon 4 and “Hide Sentiment” icon 6,although these icons could be shown anywhere on the web page. In thisembodiment, the body of the article may be divided into subportions,each subportion corresponding to a paragraph. For example, the articledisplayed in FIG. 3 may be divided into subportions 302, 304, 306, 308,310, 312, and 314.

An overall comment box 316 for commenting on the article as a whole maydisplayed below the body of the article. In some embodiments, a user mayprovide textual input (e.g., a comment) within overall comment box 316.In such a case, the user may not be targeting any particular subportionof the web page to provide sentiment for. In some embodiments, ananalysis may be performed on the user's textual input within the overallcomment box 316. For example, if the user quotes a section of thearticle, (e.g. “SIXTY-FOUR PERCENT OF AMERICANS VIEW THE REPUBLICANPARTY UNFAVORABLY” from subportion 304) an analysis may determine theuser is providing sentiment related to subportion 304. This comment maythen be assigned a tag associating it with subportion 304. In someembodiments, a direct quote may not be necessary and common keywords maybe used to associate comments with a particular subsection. For example,if textual input is entered into the overall comment box 316 thatcontains the word “Boehner,” this comment may be associated withsubportion 308, since it is the only subportion that mentions JohnBoehner. In some embodiments, if comments directed to a particularsubportion are displayed, textual input to the overall comment box 316with a tag assigning it to that subportion may also be displayed.

In some embodiments, the textual input into the overall comment box 316may also be analyzed for sentiment. If negative words like “lie,”“disagree,” “false,” etc. appear within the comment, the sentiment scoreassociated with the comment and/or subportion 304 may be decreased.Similarly, the user's history and/or profile may be analyzed. If theuser identifies himself as a Republican, the assumption may be made thatthe user does not like a subportion describing the low ratings of theRepublican Party. The sentiment score for subportion 304 may thus bedecreased. If the user's history includes many positive comments orsentiments for quotes, articles, subportions, etc. that describe lowratings for the Republican Party, it may be assumed this comment is alsopositive. The sentiment score for subportion 304 may thus be increasedbased on this user history.

FIG. 4 illustrates an excerpt of the electronic content. In this figure,the excerpt is the body of the article. A user may highlight a portionof text within the electronic content, e.g., “IT'S ONLY A 2-POINTINCREASE FROM A SURVEY TAKEN SEPT. 27 THROUGH 29” of subportion 304.When a user selects a subportion or highlights text within a subportion,a comment box, e.g. comment box 422, may be displayed proximate tosubportion 304. In the example illustrated in FIG. 4, if the userselects text within subportion 304, comment box 422 may be displayedunder subportion 304 and the user may insert textual input withincomment box 422. Display of user textual input is described below withrespect to FIG. 8.

In some embodiments, the user's textual input may be categorized orassigned tag. For example, a server may auto-suggest tags relevant tothe textual input (e.g., if the user quotes something, the server mayassign a tag related to the quote.) The server may automatically tag acomment by default. For example, if the comment recites a quote from thetext about a particular politician, e.g., John Boehner, the commentand/or subsection may automatically be tagged with a tag entitled “JohnBoehner.” In some embodiments, the user who submitted the comment orother users reading the comment may un-tag irrelevant/incorrect tags. Insome embodiments, the user may create tags for his/her own comments.Additionally or alternatively, other users may assign tags to a comment.

In some embodiments, tags may be automatically associated with oneanother. For example, if one quote is subject to many comments, many ofwhich are commonly associated with two tags, the two tags mayautomatically be associated with one another by the system. If asubsequent user sorts or selectively displays the comments by selectingthe first tag, the second tag may additionally be suggested by thesystem for display and/or sorting purposes.

In some embodiments, candidate tags may be created based upon thecomment. The confidence in these candidate tags may be increased inresponse to analyzing the user's profile and prior comments. Forexample, political persuasion, religious persuasion, etc., may be tagsgleaned from the comment, other comments the user has made, and/or theuser's profile. For example, the comment may cause the automaticcreation of a candidate tag “liberal.” However, upon review of thecandidate's profile and previous comments, the tag of “liberal” may bediscarded. Another tag, for example, “libertarian,” may also beautomatically chosen. A tag may also simply be the username, which wouldallow other users to search for comments made by that user.

FIG. 5 illustrates another embodiment of an excerpt of the electroniccontent. Specifically, FIG. 5 depicts a view of the body of an articleonce a user selects the “Show Sentiment” icon 4. It should be noted thatin some embodiments, a web page may initially be displayed with indiciaof sentiment for each subportion without user interaction with showsentiment icon 4.

A user may provide sentiment input for each subportion of the article.In the embodiment illustrated in FIG. 5, a user may provide sentimentinput by selecting the up vote icon 32 or the down vote icon 34. A usermay provide textual input by, as described with respect to FIG. 4,selecting text within a subportion or selecting any point within thesubportion. Such actions may cause a comment box to appear. In someembodiments, selecting the comment point icon 36 may cause a comment boxto appear to receive textual input from the user. Additionally oralternatively, selecting the comment point icon 36 may display previoususer comments, as further described with respect to FIG. 8 below. Asshown in FIGS. 5-8, each of subportions 502, 504, 506, 508, 510, 512,and 514 may be associated with a way of providing sentiment input and/ortextual input. The up/down vote, comment boxes, and configuration of theGUI are merely exemplary.

In the embodiment illustrated in FIG. 5, each of subportions 502, 504,506, 508, 510, 512, and 514 may have indicia based on its sentimentscore. As shown in FIG. 5, these indicia may include shading and/orcoloration, each shading/coloration indicating a sentiment score or arange of sentiment scores. The indicia may be any indication thatexpresses a range of sentiment scores, for example, various colors,number of stars, percentages, etc. In some embodiments, for example, arange of colors from green to red may encompass each subportion. A greenbox may encompass a subportion that may have a high or very positivesentiment score. Whereas a dark red box may encompass a subportion thatmay have a very negative sentiment score. As described in FIG. 2, a verynegative sentiment score may be due to, for example, multiple negativesentiment input (e.g., many down votes), strong negative sentiment input(e.g., 2% out of 100%), and/or negative textual input (e.g., multiplecomments including negative words like “hate,” “strongly disagree,”“incorrect,” etc.).

FIG. 6 illustrates another embodiment of an excerpt of the electroniccontent. Specifically, FIG. 6 depicts a view of the body of an articlewhen both indicia of sentiment and a comment box 62 are displayed to theuser. In some embodiments, a comment box may be initially displayed foreach subportion without user interaction. In some embodiments, if a userindicates a desire to comment on a specific subportion (e.g., byselecting a subportion 502, highlighting text with subportion 502, orselecting comment point icon 36) comment box 62 may be displayedproximate to said subportion.

In some embodiments, a user may make several comments about a singlesubportion. For example, as illustrated in FIG. 7, a user may makemultiple comments with respect to subportion 502. Display of each ofcomment boxes 72, 74, and 76 may be initiated in a different way. Forexample, the user may have selected comment point icon 36 to initiatedisplay of comment box 76, allowing the user to add textual inputrelated to subportion 502. Additionally or alternatively, the user mayclick/select any point within subportion 502 to initiate display ofcomment box 74, allowing the user to add textual input related tosubportion 502. In some embodiments, the user may highlight a specificsection of the subportion 502. As shown in FIG. 7, the user mayhighlight the term “THE REPUBLICAN PARTY” to initiate display of thecomment box 72 may be displayed, allowing the user to input text relatedto “THE REPUBLICAN PARTY,” and not the entirety of subportion 502.

In some embodiments, the ability to provide sentiment (e.g. sentimentinput or textual input) related to individual words or phrases within aparagraph (e.g., “THE REPUBLICAN PARTY”), may be used to calculate asentiment score for the phrase, separate from the full paragraph orsubportion. For example, subportion 508 of FIG. 7 is marked with firstindicia. In FIG. 7, the first indicia may include a first shadingencompassing subportion 508. The phrase “42 PERCENT FROM 39 PERCENT” mayhave second indicia (e.g., second shading). This second shading mayindicate that the phrase “42 PERCENT FROM 39 PERCENT” may have adifferent sentiment score associated with it than the remaining portionsof subportion 508.

In some embodiments, a user may select an indicator (e.g. click a shadedportion encompassing a subportion). The input of other users may then bedisplayed to show how/why the subportion is designated with thatindicator. For example, sentiment scores, number of comments/votes orthe actual previous use comments may be displayed.

FIG. 8 illustrates another embodiment of an excerpt of the electroniccontent. Specifically, FIG. 8 depicts a view of the body of an articlewhen both indicia of user sentiment for a plurality of subsections andprevious user comments 32, 34, and 36 are displayed to the user. Thesecomments may be displayed proximate the subportion to which they relate.Previous user comments 32, 34, and 36 may have been submitted by anyuser, including the user currently viewing the web page. The previouscomments may be in any order and displayed in any configuration.

It may be determined which subportion they relate to by tagging themwhen a user selected the subportion or terms within the subportion asdescribed in FIG. 2-7. Additionally or alternatively, comments submittedin overall comment box 316 may be tagged to relate to a specificsubportion and thus displayed proximate to that subportion. Thesecomments may be determined to be related to a subportion by analysis ofthe text, the submitting user's tags, and/or other users' tags of thecomment.

In some embodiments, previous user comments related to subportions maybe displayed initially, without user action. In some embodiments,previous user comments related to subportions may not be displayed untilsome user action is taken. For example, a user may select the commentpoint icon 36. In some embodiments, previous user comments may bedisplayed when a user “hovers” or “mouse” over a particular quote orportion of the text. This “hovering” action may result in commentspreviously tagged as relevant to the text to automatically pop up overthe text. These previously tagged comments may be sorted by default bypopularity, time made, ideological similarity with the user, etc.

In some embodiments, previous user comments may be filtered. The usercurrently viewing the web page may select a filter. In some embodiments,the web page administer, content provider, etc. may choose to filterwhich comments are displayed.

In some embodiments, previous user comments may be filtered by usersentiment. Users may provide sentiment (e.g. sentiment input, textualinput, etc.) with respect to previous user comments. In suchembodiments, a sentiment score may be calculated for each comment aswell. This calculation may be performed in a manner similar to thatdescribed with respect to web page subportions of FIG. 2. In someembodiments, a user or a web page administrator may filter all commentswith less than three stars, thus only displaying comments with asentiment score of 3 or more stars, for example.

In some embodiments, tags may be used to filter comments. Comments maybe filtered by these tags. A user or content provider may select thatonly comments associated with certain tags be displayed. The selectionmay be done by the user typing the desired tag. For example, a user maytype in a username if the user wants to see all comments by a specificuser. A user may type “liberal” to see all comments tagged as being madeby a liberal user.

A user may select a “show me similar comments” icon to automaticallysearch for comments with a similar set of tags, comments related to thesame or similar quoted portion of the content, and/or comments fromusers with a similar level of respect (e.g., users with a large numberof up votes per comment).

In some embodiments, comments may be sorted by user respect. This may bedetermined by, for example, the average number of up votes per commentthat the user has made. User respect may also be sorted by the ratio ofup votes to down votes. A higher ratio of up votes to down votes in auser's comment history may be a better indicator of comment quality thanthe sheer number of up votes.

A user may set all of these comment preferences by default. For example,a user may set rules such that only user accounts associated with a“liberal” or “moderate” tag be displayed in the comments section. A usermay also set a rule that only users with a user respect above apredetermined level be displayed. Multiple rules may also be combinedand/or prioritized. A plurality of rules and a hierarchy of sortingpriorities may also be designated for each piece of content and/or bydefault. For example, a user may designate that only comments associatedwith a “conservative” tag be displayed, sorted primarily by date, andsecondarily by user respect. These rules may be set by default for allcontent

In some embodiments, filter rules may be automatically created. Forexample, a user's comments and interactions may be monitored. Forexample, a server may monitor the tags associated with comments forwhich the user provides sentiment input (e.g., up votes and down votes),provides textual input (e.g., comments on), or otherwise interacts with.Rules may automatically be created for comment display based upon theseinteractions. For example, if a user tends to up vote comments which areassociated with a “liberal” tag and a high level of user respect, thenan automatic rule may be created which displays content to the usermatching these tags.

In some embodiments, comments may automatically be associated with othercomments quoting the same or similar portions of the content. There maybe a predetermined proximity threshold for two comments to be consideredrelated. For example, a first comment may quote or tag “four score” anda second comment may quote or tag “and seven years ago” from a passagein content that reads “four score and seven years ago.” Since the twocomments concern adjoining text, they may be associated with each otheras being within a predetermined proximity threshold.

Similarly, in some embodiments, tag associations may also be based ongrammar and punctuation. For example, comments that cite “four score”may be associated with comments that cite “seven years ago” because bothportions of the content come from the same sentence. Alternatively or inaddition, comments may be associated with each other and taggedaccordingly if they are on the same side of a comma, hyphen, semi-colon,or any other manner of associating comments by grammar and punctuation.Two portions of a content piece may be given confidence scores based onmultiple criteria, with a confidence above a predetermined thresholdcausing two comments to become associated with each other. For example,a confidence score that two comments are related may be increased basedupon shared words in the comments themselves, proximity of the quotedcontent pieces to each other, whether the quotes overlap or are in thesame sentence, etc.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments. The objects and advantages of the disclosedembodiments will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

The examples described above with respect to FIGS. 1-8, or any part(s)or function(s) thereof, may be implemented using hardware, softwaremodules, firmware, tangible computer readable media having instructionsstored thereon, or a combination thereof and may be implemented in oneor more computer systems or other processing systems.

FIG. 9 illustrates a high-level functional block diagram of an exemplarycomputer system 700, in which embodiments of the present disclosure, orportions thereof, may be implemented, e.g., as computer-readable code.For example, each of the exemplary devices and systems described abovewith respect to FIGS. 1-8 can be implemented in computer system 700using hardware, software, firmware, tangible computer readable mediahaving instructions stored thereon, or a combination thereof and may beimplemented in one or more computer systems or other processing systems.Hardware, software, or any combination of such may embody any of themodules and components in FIG. 1, as described above.

If programmable logic is used, such logic may execute on a commerciallyavailable processing platform or a special purpose device. One ofordinary skill in the art may appreciate that embodiments of thedisclosed subject matter can be practiced with various computer systemconfigurations, including multi-core multiprocessor systems,minicomputers, mainframe computers, computer linked or clustered withdistributed functions, as well as pervasive or miniature computers thatmay be embedded into virtually any device.

For instance, at least one processor device and a memory may be used toimplement the above described embodiments. A processor device may be asingle processor, a plurality of processors, or combinations thereof.Processor devices may have one or more processor “cores.”

Various embodiments of the present disclosure, as described above in theexamples of FIGS. 1-8 may be implemented using computer system 700.After reading this description, it will become apparent to a personskilled in the relevant art how to implement embodiments of the presentdisclosure using other computer systems and/or computer architectures.Although operations may be described as a sequential process, some ofthe operations may in fact be performed in parallel, concurrently,and/or in a distributed environment, and with program code storedlocally or remotely for access by single or multi-processor machines. Inaddition, in some embodiments the order of operations may be rearrangedwithout departing from the spirit of the disclosed subject matter.

As shown in FIG. 9, computer system 700 includes a central processingunit (CPU) 720. CPU 720 may be any type of processor device including,for example, any type of special purpose or a general purposemicroprocessor device. As will be appreciated by persons skilled in therelevant art, CPU 720 also may be a single processor in amulti-core/multiprocessor system, such system operating alone, or in acluster of computing devices operating in a cluster or server farm. CPU720 is connected to a data communication infrastructure 710, forexample, a bus, message queue, network, or multi-core message-passingscheme.

Computer system 700 also includes a main memory 740, for example, randomaccess memory (RAM), and may also include a secondary memory 730.Secondary memory 730, e.g., a read-only memory (ROM), may be, forexample, a hard disk drive or a removable storage drive. Such aremovable storage drive may comprise, for example, a floppy disk drive,a magnetic tape drive, an optical disk drive, a flash memory, or thelike. The removable storage drive in this example reads from and/orwrites to a removable storage unit in a well-known manner. The removablestorage unit may comprise a floppy disk, magnetic tape, optical disk,etc. which is read by and written to by the removable storage drive. Aswill be appreciated by persons skilled in the relevant art, such aremovable storage unit generally includes a computer usable storagemedium having stored therein computer software and/or data.

In alternative implementations, secondary memory 730 may include othersimilar means for allowing computer programs or other instructions to beloaded into computer system 700. Examples of such means may include aprogram cartridge and cartridge interface (such as that found in videogame devices), a removable memory chip (such as an EPROM, or PROM) andassociated socket, and other removable storage units and interfaces,which allow software and data to be transferred from a removable storageunit to computer system 700. Control system 700 may receive programmingand data via network communications 970.

Computer system 700 may also include a communications interface (“COM”)760. Communications interface 760 allows software and data to betransferred between computer system 700 and external devices.Communications interface 760 may include a modem, a network interface(such as an Ethernet card), a communications port, a PCMCIA slot andcard, or the like. Software and data transferred via communicationsinterface 760 may be in the form of signals, which may be electronic,electromagnetic, optical, or other signals capable of being received bycommunications interface 760. These signals may be provided tocommunications interface 760 via a communications path of computersystem 700, which may be implemented using, for example, wire or cable,fiber optics, a phone line, a cellular phone link, an RF link or othercommunications channels.

The hardware elements, operating systems and programming languages ofsuch equipment are conventional in nature, and it is presumed that thoseskilled in the art are adequately familiar therewith. Computer system700 also may include input and output ports 750 to connect with inputand output devices such as keyboards, mice, touchscreens, monitors,displays, etc. Of course, the various server functions may beimplemented in a distributed fashion on a number of similar platforms,to distribute the processing load. Alternatively, the servers may beimplemented by appropriate programming of one computer hardwareplatform.

Program aspects of the technology may be thought of as “products” or“articles of manufacture” typically in the form of executable codeand/or associated data that is carried on or embodied in a type ofmachine readable medium. “Storage” type media include any or all of thetangible memory of the computers, processors or the like, or associatedmodules thereof, such as various semiconductor memories, tape drives,disk drives and the like, which may provide non-transitory storage atany time for the software programming. All or portions of the softwaremay at times be communicated through the Internet or various othertelecommunication networks. Such communications, for example, may enableloading of the software from one computer or processor into another, forexample, from a management server or host computer of the mobilecommunication network into the computer platform of a server and/or froma server to the mobile device. Thus, another type of media that may bearthe software elements includes optical, electrical and electromagneticwaves, such as used across physical interfaces between local devices,through wired and optical landline networks and over various air-links.The physical elements that carry such waves, such as wired or wirelesslinks, optical links or the like, also may be considered as mediabearing the software. As used herein, unless restricted tonon-transitory, tangible “storage” media, terms such as computer ormachine “readable medium” refer to any medium that participates inproviding instructions to a processor for execution.

It would also be apparent to one of skill in the relevant art that thepresent disclosure, as described herein, can be implemented in manydifferent embodiments of software, hardware, firmware, and/or theentities illustrated in the figures. Any actual software code with thespecialized control of hardware to implement embodiments is not limitingof the detailed description. Thus, the operational behavior ofembodiments will be described with the understanding that modificationsand variations of the embodiments are possible, given the level ofdetail presented herein.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

Other embodiments of the disclosure will be apparent to those skilled inthe art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the invention being indicated by the following claims.

What is claimed is:
 1. A computer-implemented method for evaluating userinput relating to electronic published content, the method comprising:receiving, over an electronic network, electronic publishing content fordisplay online; displaying the received publishing content on a web pagealong with one or more user elements by which one or more users maysubmit sentiment input or textual input in relation to the receivedpublishing content or a subportion of the received publishing content;receiving, from each of a plurality of users, a user input related tothe displayed publishing content displayed on the web page, the userinput including an identification of a subportion of the displayedpublishing content and a sentiment input or a textual input; analyzing,for each subportion of the displayed publishing content, any user inputreceived from each of one or more of the plurality of users in relationto the subportion; computing, for each subportion of the displayedpublishing content, a sentiment score based on analysis of the analyzeduser inputs received from each of one or more of the plurality of usersin relation to the subportion; and displaying, for each subportion ofthe displayed publishing content, indicia representing the sentimentscore computed for the subportion, along with at least one user elementby which one or more further users are enabled to provide user input tofurther modify the computed and indicated sentiment score.
 2. Thecomputer-implemented method of claim 1, wherein the textual input is auser comment.
 3. The computer-implemented method of claim 1, wherein theuser input further comprises one or more tags received from the user inrelation to one or more subportions of the publishing content.
 4. Thecomputer-implemented method of claim 1, wherein the user input is asentiment input or a textual input.
 5. The method of claim 1, furthercomprising: dividing the received electronic publishing content into aplurality of subportions.
 6. The computer-implemented method of claim 1,wherein the sentiment score is associated with at least one of theplurality of subportions.
 7. The computer-implemented method of claim 4,further comprising: changing an indicator associated with the at leastone subportion based on the modified sentiment score.
 8. A computersystem for evaluating user input relating to electronic publishedcontent, the system comprising: a memory device storing instructions forevaluating user input; and a processor configured to execute theinstructions to perform a method of: receiving, over an electronicnetwork, electronic publishing content for display online; displayingthe received publishing content on a web page along with one or moreuser elements by which one or more users may submit sentiment input ortextual input in relation to the received publishing content or asubportion of the received publishing content; receiving, from each of aplurality of users, a user input related to the displayed publishingcontent displayed on the web page, the user input including anidentification of a subportion of the displayed publishing content and asentiment input or a textual input; analyzing, for each subportion ofthe displayed publishing content, any user input received from each ofone or more of the plurality of users in relation to the subportion;computing, for each subportion of the displayed publishing content, asentiment score based on analysis of the analyzed user inputs receivedfrom each of one or more of the plurality of users in relation to thesubportion; and displaying, for each subportion of the displayedpublishing content, indicia representing the sentiment score computedfor the subportion, along with at least one user element by which one ormore further users are enabled to provide user input to further modifythe computed and indicated sentiment score.
 9. The computer system ofclaim 8, wherein the textual input is a user comment.
 10. The computersystem of claim 8, wherein the user input further comprises one or moretags received from the user in relation to one or more subportions ofthe publishing content.
 11. The computer system of claim 8, wherein thesentiment score is assigned to one of a sentiment input and a textualinput.
 12. The computer system of claim 8, further comprising: dividingthe received electronic publishing content into a plurality ofsubportions.
 13. The computer system of claim 8, wherein the sentimentscore is associated with at least one of the plurality of subportions.14. The computer system of claim 13, further comprising: changing anindicator associated with the at least one subportion based on themodified sentiment score.
 15. A non-transitory computer-readable mediumstoring instructions, then instructions, when executed by a computersystem cause the computer system to perform a method, the methodcomprising: receiving, over an electronic network, electronic publishingcontent for display online; displaying the received publishing contenton a web page along with one or more user elements by which one or moreusers may submit sentiment input or textual input in relation to thereceived publishing content or a subportion of the received publishingcontent; receiving, from each of a plurality of users, a user inputrelated to the displayed publishing content displayed on the web page,the user input including an identification of a subportion of thedisplayed publishing content and a sentiment input or a textual input;analyzing, for each subportion of the displayed publishing content, anyuser input received from each of one or more of the plurality of usersin relation to the subportion; computing, for each subportion of thedisplayed publishing content, a sentiment score based on analysis of theanalyzed user inputs received from each of one or more of the pluralityof users in relation to the subportion; and displaying, for eachsubportion of the displayed publishing content, indicia representing thesentiment score computed for the subportion, along with at least oneuser element by which one or more further users are enabled to provideuser input to further modify the computed and indicated sentiment score.16. The non-transitory computer-readable medium of claim 15, wherein thetextual input is a user comment.
 17. The non-transitorycomputer-readable medium of claim 15, wherein the user input furthercomprises one or more tags received from the user in relation to one ormore subportions of the publishing content.
 18. The non-transitorycomputer-readable medium of claim 15, wherein the sentiment score isassigned to one of a sentiment input and a textual input.
 19. Thenon-transitory computer-readable medium of claim 15, further comprising:dividing the received electronic publishing content into a plurality ofsubportions.
 20. The non-transitory computer-readable medium of claim15, wherein the sentiment score is associated with at least one of theplurality of subportions.